Papers with NAG model
Non-Autoregressive Text Generation with Pre-trained Language Models (2021.eacl-main)
Copied to clipboard
| Challenge: | Autoregressive generation models generate tokens in a left-to-right, token-by-token fashion, resulting in lag in inference. |
| Approach: | They propose to use BERT as the backbone of a non-autoregressive generation model for greatly improved performance. |
| Outcome: | The proposed model outperforms existing non-autoregressive models and achieves competitive performance with many strong autoregressive model. |
POS-Constrained Parallel Decoding for Non-autoregressive Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing non-autoregressive generation systems face multimodality problem due to conditionally independent decoding. |
| Approach: | They propose to incorporate linguistic structure into NAG inference instead of teacher AG . they propose a method that provides a specific POS sequence to constrain the NAG model . |
| Outcome: | The proposed method improves NAG models on four text generation tasks to a greater extent compared to knowledge distillation. |